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Abstract 

Environmental signals sensed by nervous systems are often represented 
in spike trains carried from sensory neurons to higher neural functions 
where decisions and functional actions occur. Information about the en- 
vironmental stimulus is contained (encoded) in the train of spikes. We 
show how to "read" the encoding using state space methods of nonlin- 
ear dynamics. We create a mapping from spike signals which are output 
from the neural processing system back to an estimate of the analog in- 
put signal. This mapping is realized locally in a reconstructed state space 
embodying both the dynamics of the source of the sensory signal and the 
dynamics of the neural circuit doing the processing. We explore this idea 
using a Hodgkin-Huxley conductance based neuron model and input from 
a low dimensional dynamical system, the Lorenz system. We show that 
one may accurately learn the dynamical input/output connection and esti- 
mate with high precision the details of the input signals from spike timing 
output alone. This form of "reading the neural code" has a focus on the 
neural circuitry as a dynamical system and emphasizes how one interprets 
the dynamical degrees of freedom in the neural circuit as they transform 
analog environmental information into spike trains. 

1 Introduction 

A primary task of nervous systems is the collection at its periphery of informa- 
tion from the environment and the distribution of that stimulus input to central 
nervous system functions. This is often accomplished through the production 
and transmission of action potentials or spike trains 16 . 

The book JS| and subsequent papers by its authors and their collabora- 
tors [2] carefully lay out a program for interpreting the analog stimulus of a 
nervous system using ideas from probability theory and information theory, as 
well as a representation of the input/output or stimulus/response relation in 
terms of Volterra kernel functions. In the authors note that when present- 
ing a stimulus to a neuron, it is common "that the response spike train is not 
identical on each trial." Also they observe that "Since there is no unique re- 
sponse, the most we can say is that there is some probability of observing each 
of the different possible responses." This viewpoint then underlies the wide use 
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of probabilistic ideas in describing how one can "read the neural code" through 
interpreting the response spike trains to infer the stimulus. 

In this paper we take a different point of view and recognize that the neuron 
into which one sends a stimulus is itself a dynamical system with a time depen- 
dent state which will typically be different upon receipt of different realizations 
of identical stimulus inputs. Viewing the transformation of the stimulus wave- 
form into the observed response sequence, as a result of deterministic dynamical 
action of the neuron one can attribute the variation in the response to identical 
stimuli to differing neuron states when the stimulus arrives. This allows us to 
view the entire transduction process of analog input (stimulus) to spike train 
output (response) as a deterministic process which can be addressed by methods 
developed in nonlinear dynamics for dealing with input/output systems |14| . 

Previous research on information encoding in spike trains has concentrated 
on nonlinear filters that convert analog input signals into spike trains. It has 
been shown that these models can be used to reconstruct the dynamical phase 
space of chaotic inputs to the filters using the spike timing information [171 
^iniMj- Using simple dynamical neuron models, Castro and Sauer |S] have 
shown that aspects of a dynamical system can be reconstructed using interspike 
intervals (ISIs) properties. Experimental work has demonstrated the ability to 
discriminate between chaotic and stochastic inputs to a neuron |15|. as well as 
showing that decoding sensory information from a spike train through linear 
filtering Volterra series techniques can allow for large amounts of information 
to be carried by the precise timing of the spikes HQ . 

We discuss here the formulation of input / output systems from a dynamical 
system point of view, primarily summarizing earlier work jl4l [T]. but with a fo- 
cus on recognizing that we may treat the response signals as trains of identical 
spikes. Since the modulation of the spike train must be carrying the informa- 
tion in the analog input presented to the neuron, if the spike pulse shapes are 
identical, all information must be encoded in the ISIs. We shall show that this 
is, indeed, the case. 

What is the role of information theory in a deterministic chain of actions from 
stimulus to spiking response? The ideas of information theory, though often 
couched in terms of random variables, applies directly to distributed variation 
in dynamical variables such as the output from nonlinear systems. The use of 
concepts such as entropy and mutual information, at the basis of information 
theoretic descriptions of systems, applies easily and directly to deterministic 
systems. The understanding of this connection dates from the 1970's and 1980's 
where the work of Fraser j6j makes this explicit, and the connection due to 
Pesin 22 between positive Lyapunov exponents of a deterministic system and 
the Kolmogorov-Sinai entropy quantifies the correspondence. 

In the body of this paper, we first summarize the methods used to determine 
a connection between analog input signals and spiking output, then we apply 
these methods to a Hodgkin- Huxley conductance based model of the R15 neuron 
of Aplysia |31 E| • Future papers will investigate the use of these methods on 
biological signals from the HI visual neuron of a fly and a stretch receptor in 
the tail of a crayfish [201 
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2 Input Estimation from State Space Reconstruc- 
tion 



The general problem we address is the response to stimuli of a neural circuit 
with N dynamical variables 

x(t) = [xi{t),X2{t), . . . ,XNit)]. 

When there is no time varying input, x(i) satisfies the ordinary differential 
equations 

^^^F4^{t)), ;a=l,2,...,iV (1) 

The Fa (x) are a set of nonlinear functions which determine the dynamical time 
course of the neural circuit. The Fa(x) could well represent a conductance based 
neural model of the Hodgkin-Huxley variety as in our example below. 

When there is a time dependent external stimulus s(t), these equations be- 
come 

^ = F4xW,.(i)), (2) 

and the time course of x(t) in this driven or non-autonomous setting can become 
rather more complicated than the case where s{t) — constant. 

If we knew the dynamical origin of the signal s{t), then in the combined 
space of the stimuli and the neural state space x(i), we would again have an 
autonomous system, and many familiar 1 methods for analyzing signals from 
nonlinear systems would apply. As we proceed to our "input signal from spike 
outputs" connection we imagine that the stimulus system is determined by some 
other set of state variables z(t) and that 

= G(z(t)) 

s{t) - /i(z(t)), (3) 

where G(z) are the nonlinear functions determining the time course of the state 
z(i) and h{z{t)) is the nonlinear function determining the input to the neuron 
sit). 

With observations of just one component of the state vector x(t), the full 
dynamical structure of a system described by Equation[21can be reconstructed in 
a proxy state space |51 119). Once the dynamics of the system is reconstructed, 
the mapping from state variable to input can be made in the reconstructed 
space. Assume the measured state variable, r{t) — g(x(i)), is sampled at times 
tj, where j is an integer index. According to the embedding theorem |^ I19|. 
the dynamics of the system can be reconstructed in an embedding space using 
time delayed vectors of the form 

y(i) = [r{tj),r{t^ + Tr,), . . . , r{t^ + {ds - 1)Tt,)] 

= [r{j),r{j+T),...,rij + {dE-l)T)] (4) 
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where is the dimension of the embedding, tj ~ to + jTs, Tg is the sampUng 
time, to is an initial time, and T is an integer time delay. If the dimension (Ie is 
large enough these vectors can reconstruct the dynamical structure of the full 
system given in Equation |21 Each vector y(j) in the reconstructed phase space 
depends on the state of the input signal. Therefore a mapping should exist that 
associates locations in the reconstructed phase space y(j) to values of the input 
signal s{tj) = s{i) : s{j) ~ H{y{j)). The map H{y) is the output-to-input 
relation we seek. 

Without simultaneous measurements of the observable r{t) and the input 
signal s{t), this mapping could not be found without knowing the differential 
equations that make up Equation |21 But in a situation where a controlled 
stimulus is presented to a neuron while measuring the output, both r{t) and s{t) 
are available simultaneously. Such a data set with simultaneous measurements 
of spike time and input is split into two parts: the first part, called the training 
set, will be used to find the mapping H{y{j)) between y(j) and s(j). The second 
part, called the test set, will be used to test the accuracy of that mapping. State 
variable data from the training set r{j) is used to construct time delayed vectors 
as given by 

y(j) = [rij),r{j + T), . . . , r(j + (d^ - l)r)]. (5) 

Each of these vectors is paired with the value of the stimulus at the midpoint 
time of the delay vector 

^(j) = « (^+T(di5-l)/2) (6) 

We use state space values that occur before and after the input to improve the 
quality of the representation. The state variables and input values in the re- 
mainder of the data are organized in a similar way and used to test the mapping. 

The phase space dynamics near a test data vector are reconstructed using 
vectors in the training set that are close to the test vector, where we use Eu- 
clidian distance between vectors. These vectors lie close in the reconstructed 
phase space, so they will define the dynamics of the system in that region and 
will define a local map from that region to a input signal value. In other words, 
we seek a form for H{y{j)) which is local in reconstructed phase space to y(j). 
The global map over all of phase space is a collection of local maps. 

The local map is made using the Nb nearest neighbors y™(j ) , m — . . . Ng 
of y°(i) = y(j)- These nearest neighbor vectors and their corresponding input 
values s"^{j) are used to find a local polynomial mapping between inputs s"^{j) 
and vector versions of the outputs namely y™(j) of the form 

= Hiy"\j)) = Moij)+M,ij)-r^u)+M,ij)-r^u)-r^u) +■■■, (?) 

which assume that the function H{y) is locally smooth in phase space. 

The scalar Mo{j), the d_E-dimensional vector Mi(j), and the tensor M.2{j) 
in de-dimensions, etc are determined by minimizing the mean squared error 

Nb 

- Moij) + M,U) ■ y"\j) + M,{j) ■ y"\j) ■ y"\j) + ■ • ■ p. (8) 

m=0 
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We determine Mo{j), Mi(j), M2(j), ... for all j = 1, 2, . . ., and this provides a 
local representation of H{y) in all parts of phase space sampled by the training 

set y(j), j = 1,2, . ..,Ntrain- 

Once the least squares fit values of Mo(j), Mi(j), M2(j), . . . are determined 
for our training set, we can use the resulting local map to determine estimates of 
the input associated with an observed output. This proceeds as follows: select a 
new output r"'^'^{l) and form the new output vector y"^'"(Z) as above. Find the 
nearest neighbor in the training set to y"^"'(/). Suppose it is the vector y{q). 
Now evaluate an estimated input s'^''*(^) as 

= Mo{q) + Mi(g) • y"«-(0 + M^iq) ■ y"^"'(0 • y"™(0 + • • • . (9) 

This procedure is applied for all new outputs to produce the corresponding 
estimated inputs. 



3 R15 Neuron Model 

To investigate our ability to reconstruct stimuli of analog form presented to 
a realistic neuron from the spike train output of that neuron, we examined a 
detailed model of the R15 neuron in Aplysia and presented this model 

neuron with nonpcriodic input from a low dimensional dynamical system. This 
model has seven dynamical degrees of freedom. The differential equations for 
this model are 

C^^^ = [gmitfy^it) + grKVi - V{t))+gL{VL - V{t)) 
+ [gKViitf + 9AV5{t)v(,{t) + gpyiimVK - V{t)) 

(10) 

where the yn{t)', n — 2,3, ... ,7 satisfy kinetic equations of the form 

dt Tn{V^{t)) ' ^ ' 

which is the usual form of Hodgkin-Huxlcy models. The gx,X — I,T, K, A, P, L 
are maximal conductances, the Vx,X = I,L,K are reversal potentials. Vm,{t) 
is the membrane potential, C is the membrane capacitance, Iq is a fixed DC 
current, and Igxt is a DC current we vary to change the state of oscillation of the 
model. The functions Yn{V) and Tn{V) and values for the various constants are 
given in 0^]. These are phenomenological forms of membrane voltage depen- 
dent gating variables, activation and inactivation of membrane ionic channels, 
and time constants for these gates. Imput {t) is a time varying current input to 
the neural dynamics. Our goal will be to reconstruct Iinput{t) from observations 
of the spike timing in Vm (t) . 

In Figure^we plot the bifurcation diagram of our R15 model. On the vertical 
axis we show the values of ISIs taken in the time series for Vm (t) from the model; 
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on the horizontal axis we plot lext- From this bifurcation plot we see that the 
output of the R15 model has regular windows for lext < -07 then chaotic regions 
interspersed with periodic orbits until lext ~ 0.19 after which nearly periodic 
behavior is seen. The last region represents significant depolarization of the 
neuron in which tonic periodic firing associated with a stable limit cycle in 
phase space is typical of neural activity. Periodic firing leads to a fixed value 
for ISIs, which is what we see. Careful inspection of the time series reveals very 
small fluctuations in the phase space orbit, but the resolution in Figure ^ does 
not expose this. 

Other than the characteristic spikes, there are no significant features in the 
membrane voltage dynamics. In addition all the spikes are essentially the same, 
so we expect that all the information about the membrane voltage state is 
captured in the times between spikes, namely the interspikc intervals: ISIs. The 
distribution of ISIs characterizes the output signal for information theoretic 
purposes. 

We have chosen three values of lext at which to examine the response of this 
neuron model when presented with an input signal. At lext = 0.1613 we expect 
chaotic waveforms expressed as nonperiodic ISIs with a broad distribution. At 
lext = 0.2031 we expect nearly periodic spike trains. And at lext — —0.15 the 
neuron does not spike, the mebrane voltage remains at an equilibrium value. 

For each Vm{t) time series we evaluate the normalized distribution of ISIs 
which we call P/s/(A) and from this we compute the entropy associated with 
the oscillations of the neuron. Entropy is defined as 



H{A) > 0. The entropy is a quantitative measure |^ of the information content 
of the output signal from the neural activity. 

In Figure 121 we display a section of the Vm{t) time series for lext = 0.1613. 
The irregularity in the spiking times is clear from this figure and the distribution 
P/S/(A) shown in FigureOl The P/s/(A) was evaluated from collecting 60,000 
spikes from the Vm{t) time series and creating a histogram with 15,000 bins. 
This distribution has an entropy H{A) = 12. In contrast to this we have a 
section of the Vm{t) time series for lext = 0.2031 in Figure^ Far more regular 
firing is observed with a firing frequency much higher than for lext — 0.1613. 
This increase in firing frequency as a neuron is depolarized is familiar. With 
lext = 0.2031 the distribution Pjsj(A) is mainly concentrated in one bin with 
some small flucuations near that bin. Such a regular distribution leads to a very 
low entropy ^^(A) = 0.034. If not for the slight variations in ISI, the entropy 
would be zero. If P/5/(Ao) = 1 for some ISI value Aq, then H{A) ~ 0. 

3.1 Input Signals to Model Neuron 

In the last section the dynamics of the neuron model were examined using con- 
stant input signals. In studying how neurons encode information in their spike 
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Figure 1: Bifurcation diagram for the R15 model with constant input current. 
This plot shows the values of ISIs which occur in the V„i{t) time series for 
different values of lext ■ 



train, we must clarify what it means for a signal to carry information. In the 
context of information theory fIB) . information lies in the unpredictability of a 
signal. If we do not know what a signal is going to do next, then by observing it 
we gain new information. Stochastic signals are commonly used as information 
carrying signals since their unpredictability is easily characterized and readily 
incorporated into the theoretical structure of information theory. But they are 
problematic when approaching a problem from a dynamical systems point of 
view, since they are systems with a high dimension. This means that the recon- 
struction of a stochastic signal using time delay embedding vectors of the form 
of Equation 01 would require an extremely large embedding dimension pp. If we 
are injecting stochastic signals into the R15 model, the dimension of the whole 
system would increase and cause practical problems in performing the input 
reconstruction. Indeed, the degrees of freedom in the stochastic input signal 
could well make the input/output relationship we seek to expose impossible to 
see. 

An attractive input for testing the reconstruction method will have some 
unpredictability but have few degrees of freedom. If there are many degrees 
of freedom, the dimensionality of the vector of outputs y(j) above may be 



7 



30 




20000 40000 60000 

Time (arb. units) 

Figure 2: Membrane voltage of the R15 model with a constant input current 
lext = 0.1613. 



prohibitively large. This leads directly to the consideration of low dimensional 
chaotic systems. Chaos originates from local instabilities which cause two points 
initially close together in phase space to diverge rapidly as the system evolves 
in time, thus producing completely different trajectories. This exponential di- 
vergence is quantified by the positive Lyapunov exponents and is the source of 
the unpredictability in chaotic systems 1 . The state of any observed system 
is known only to some degree of accuracy, limited by measurement and sys- 
tematic errors. If the state of a chaotic system were known exactly then the 
future state of that system should be exactly predictable. But if the state of a 
chaotic system is only known to some finite accuracy, then predictions into the 
future based on the estimated state will diverge from the actual evolution of the 
system. Imperfect observations of a chaotic signal will limit the predictability 
of the signal. Since chaos can occur in low dimensional systems these signals do 
not raise the same concerns as stochastic signals. 

We use a familiar example of a chaotic system, the Lorenz attractor [7], as 
the input signal to drive the R15 model. This is a well studied system that 
exhibits chaotic dynamics and will be used here as input to the R15 neuron 
model. The Lorenz attractor is defined by the differential equations 

= cr(yW-2;(i)) 

«^ - -x{t)z{t)+rx{t)-y{t) (13) 
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Figure 3: Normalized distribution Pjsi{A) from the membrane voltage time 
series with lext = 0.1613. The entropy for this distribution iJ(A) = 12. 



K— ^ = x{t)v{t) - hz{t) 

For the simulations presented in this paper the parameters were chosen as u = 
16, r — 45.92 and h — A. The parameter k is used to change the time scale. An 
example times series of the x{t) component of the Lorenz attractor is shown in 
Figure El 

3.2 Numerical Results 

An input signal s{t) — Imputit) is now formed from the x{t) component of the 
Lorenz system. Our goal is to use observations of the stimulus Iinput{t) and 
of the ISIs of the output signal Vm{t) to learn the dynamics of the R15 neuron 
model in the form of a local map in phase space reconstructed from the observed 
ISIs. From this map we will estimate the input Iinput{t) from new observations 
of the output ISIs. 

Our analog signal input is the x{t) output of the Lorenz system, scaled and 
offset to a proper range, and then input to the neuron as an external current 

hnput (t) = Amp(a;(<) +xq), (14) 
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Figure 4: Membrane voltage of the R15 model with a constant input current 
/ext = 0.2031. 



where Amp is the scaling constant and xq is the offset. The R15 equations are 
integrated p^3) with this input signal and the spike times tj from the membrane 
voltage are recorded simultaneously with the value of the input current at that 
time linput (tj ) . Reconstruction of the neuron plus input phase space is done by 
creating time delay vectors from the ISIs 

y(i) = [isij,isij+i, . . .,isi^^i^^^_^^^] (15) 

where 

isij = tj - tj-i (16) 

For each of these vectors there is a corresponding value of the input current 
which we chose to be at the midpoint time of the vector 

In our work a total of 40000 spikes were collected. The first 30000 were used to 
create the training set vectors and the next 10000 were used to examine our input 
estimation methods. For each new output vector constructed from new observed 
ISIs, Nb nearest neighbors from the training set were used to generate a local 
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Figure 5: Small segment of the x{t) component of the Lorenz attractor described 
in equations El with k = 10^. 



polynomial map y(j) — » I'input"'^'^'^ ij) ■ was chosen to be twice the number 
of free parameters in the undetermined local coefficients Afp, Mi, M2, . . .. 

We used the same three values of I^xt -0.15, 0.1613, and 0.2031 employed 
above in our simulations. We took Amp — 0.001, n = 10"*, and xo = 43.5 
for all simulations unless stated otherwise. This very small amplitude of the 
input current is much more of a challenge for the input reconstruction than 
large amplitudes. When Amp is large, the neural activity is entrained by the 
input signal and 'recovering' the input merely requires looking at the output 
and scaling it by a constant. Further, the intrinsic spiking of the neuron which 
is its important biological feature goes away when Amp is large. The large value 
of K assures that the spikes sample the analog signal Iinput{t) very well. 

For /ea;f = 0.1613 we show a selection of both the input current linput and the 
output membrane voltage Vmit) time series in Figure |S1 The injected current 
substantially changes the pattern of firing seen for the autonomous neuron. Note 
that the size of the input current is numerically about 10^"^ of Vm{t), yet the 
modulation of the ISIs due to this small input is clearly visible in Figure 

Using the ISIs of this time series we evaluated Pisi{^) as discussed above 
and from that the entropy H{A) associated with the driven neuron. The ISI 
distribution, P/5/(A), shown in Figured has an entropy H{A) = 8.16. The 
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Figure 6: A segment of the R15 neuron model output Vm{t) shown along with 
the scaled Lorenz system input current linput- Here lext — 0.1613, Amp = 0.001, 
and K = 10^. Note the different scales for linput (shown on the left axis) and 
Vm{t). (shown on the right axis) 



effect of the input current has been to substantially narrow the range of ISIs 
seen in Vm{t). This can be seen by comparison with Figure 01 

Figure |S1 shows an example of input signal reconstruction which estimates 
linput using ISI vectors of the described in Equation ^1 We used a time delay 
T = 1, an embedding dimension ds = 7, and a local linear map for H{y{j)). The 
RMS error over the 10,000 reconstructed values of the input was a = 4.6 • 10~^. 
The input signal is only reconstructed at times at which the neuron spikes. So 
each point is the reconstruction curve in Figure |H1 corresponds to a spike in 
Vmit). Some features of the input are missed because no spikes occur during 
that time, but otherwise the reconstruction is very accurate. At places where 
the spike rate is high, interpolation seems to fill the gaps between spikes. 

Different values of embedding dimension, time delay, and map order will 
lead to different reconstruction errors. For example, low embedding dimension 
may not unfold the dynamics and linear maps may not be able to fit some 
neighborhoods to the input. For the results shown here, there is little difference 
in the RMS reconstruction error if the embedding dimension is increased or 
quadratic maps are used instead of linear maps. This may not be true if lower 
embedding dimension is used. 

The previous example probed the response of a chaotic neural oscillation to 
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Figure 7: P/5/(A) for R15 model neuron output when a scaled x{t) signal 
from the Lorenz system is presented with Ig^t — 0.1613. The entropy of this 
distribution H{A) = 8.16. 



a chaotic signal. With I^xt = 0.2031 the neuron is in a periodic spiking regime 
and the input modulates the instantaneous firing rate of the neuron. A sample of 
the input current and membrane voltage is shown in Figure El The distribution 
of ISIs, P/s/(A), shown in Figure [TUI and has an entropy H{A) = 9.5. The 
effect of the input current is to substantially broaden the range of ISIs and 
increase its entropy as compared to the nearly periodic firing of the autonomous 
neuron with I^xt = 0.2031. The high spiking rate and close relationship between 
input current amplitude and ISI lead to very accurate reconstructions using low 
dimensional embeddings. A sample of the reconstruction using = 2 and 
T = 1 is shown in Figure ITTI The RMS reconstruction error of cr = 6.1 • 10^"* 
with a maximum error of 0.007. 

In a final example we show the reconstruction when the neuron is being 
driven with an input current below the threshold for spikes. With le^t = —0.15, 
the autonomous R15 neuron will remain at an equilibrium level and not pro- 
duce spikes. A Lorenz input injected into the neuron with Amp = 0.002 and 
xo = 43.5 is large enough to cause the neuron to spike. Figure ^1 shows a 
sample of the membrane voltage time series along with the corresponding input 
current. Since the spiking rate of the neuron is much lower than before, k is 
increased to 2 • 10^. This slows down the dynamics of the Lorenz input relative 
to the neuron dynamics. Spikes occur during increasing portions of the input 
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Figure 8: ISI Reconstruction of the input Lorenz signal to an R15 neuron. The 
solid line is the actual input to the neuron. The dots joined by dashed lines are 
the ISI reconstructions. The embedding dimension of the reconstruction ds is 
4, the time delay T is 1, lext — 0.1613, k is 10^, and a linear map was used. 
The RMS error of the estimates over 10,000 estimations is cr = 4.6 • lO^'* and 
the maximum error is about 0.01. 



current and are absent for low values of input current. Figure 1131 shows the 
distribution of ISIs which has an entropy H{A) = 5.3. The low spiking rate 
shows up in the distribution in the form large numbers of long ISI. For the re- 
construction of the input larger embedding dimensions were needed. An sample 
of the reconstruction is shown in Figure ITU using cIe — 7 and T = 1. For this fit 
the RMS reconstruction error a = 0.0094 with a maximum error of 0.03. These 
errors are noticeably higher than the previous two examples. 

The accuracy of the reconstruction method depends on a high spiking rate 
in the neuron relative to the time scale of the input signal, since only one 
reconstructed input value is generated for each spike. If the spiking rate of the 
neuron is low relative to the time scales of the input signal, then the neuron will 
undersample the input signal and miss many of its features. This limitation can 
be demonstrated by decreasing the time scale parameter k, thereby speeding 
up the dynamics of the input. During the longer ISIs the input current can 
change by large amounts. Though the reconstruction undersamples the input, 
but interpolation can fill in some of the gaps. As k is increased further the 
reconstruction will further degrade. 
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Figure 9: A segment of the R15 neuron model output Vm{t) shown along with 
the scaled Lorenz system input current Imput- Here lext — 0.2031, Amp = 0.001, 
and K = 10^. Note the different scales for Imput and Vm{t)- 



4 Discussion 

In previous research on the encoding of chaotic attractors in spikes trains, the 
spike trains were produced by nonlinear transformations of chaotic input sig- 
nals. Threshold crossing neuron models have been used, which generate the 
spike times at upward crossings of a threshold. This is equivalent to a Poincare 
section of the input signal. Also integrate and fire neurons have been studied, 
which integrate the input signal and fire a spike when it crosses a threshold, after 
which the integral is reset to zero. Both of these models have no intrinsic com- 
plex dynamics; they can not produce entropy autonomously. All of the complex 
behavior is in the input signal. Even though the attractor of a chaotic input can 
be reconstructed from the ISIs, these models do not account for the complex 
behavior of real neurons. The input reconstruction method we have presented 
here allows for complex intrinsic dynamics of the neuron. We have shown that 
the local polynomial representations of input /output relations realized in recon- 
structed phase space can extract the chaotic input from the complex interaction 
between the input signal and neuron dynamics. 

Other experimental works have used linear kernels to map the spike train 
into the input. They have shown that the precise timing of individual spikes can 
encode a lot of information about the input (16) . And the precise relative tim- 
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Figure 10: Pjsi{A) for R15 model neuron output when a scaled x{t) signal 

from the Lorenz system is presented with I^xt — 0.2031. The entropy of this 
distribution H{A) = 9.5. 



ing between two spikes can carry even more information than their individual 
timings combined 0. These results may be pointing toward a state space rep- 
resentation since the time delay embedding vectors used here take into account 
both the precise spike timing and the recent history of ISIs. From a dynamical 
systems perspective this is important because the state of the system at the 
time of the input will affect its response. This is a factor that linear kernels do 
not take into account. 

The advantage of using local representations of input/output relations in 
reconstructed state space lies primarily in the insight it may provide about 
the underlying dynamics of the neural transformation process mapping analog 
environmental signals into spike trains. The goal of the work presented here is 
not primarily to show we can accurately recover analog input signals from the 
ISIs of spike output from neurons, though that is important to demonstrate. The 
main goal is to provide clues on how one can now model the neural circuitry 
which transforms these analog signals. The main piece of information in the 
work presented here lies in the size of the reconstructed space cIe which tells 
us something about the required dimension of the neural circuit. Here we see 
that a low dimension can give excellent results indicating that the complexity of 
the neural circuit is not fully utilized in the transformation to spikes. Another 
suggestion of this is in the entropy of the input and output signals. In the case 
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Figure 11: ISI Reconstruction of the input Lorenz signal to an R15 neuron. The 
solid line is the actual input to the neuron. The dots joined by dashed lines are 
the ISI reconstructions. The embedding dimension of the reconstruction ds is 
2, the time delay T is 1, lext = 0.2031, k is 10^, and a linear map was used. 
The RMS error of the estimates over 10,000 estimations is cr = 6.1 • lO^'* and 
the maximum error is about 0.007. 



where I^xt — 0.1613 the entropy of the analog input is 11.8 while the entropy of 
the ISI distribution of the output is 8.16. When lext — 0.2031 the output entropy 
is 9.5. This suggests, especially in the case of the larger current, that the signal 
into R15 neuron model acts primarily as a modulation on the ISI distribution. 
This modulation may be substantial, as in the case when I^xt = 0.1613 but 
reading the modulated signal does not require complex methods. 

Our final example took lext — —0.15 at which value the undriven neuron has 
Vm(t) = constant, so it is below threhold for production of action potentials. In 
this case the introduction of the stimulus drove the neuron above this threshold 
and produced a spike train which could be accurately reconstructed. This ex- 
ample is relevant to the behavior of biological neurons which act as sensors for 
various quantities: visiual stimuli, chemical stimuli (olfaction), etc. In the study 
of biological sensory systems [201 the neural circuitry is quiet in the absence of 
input signals, yet as we now see the methods are equally valid and accurate. 
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Figure 12: A segment of the R15 neuron model output Vm{t) shown along with 
the scaled Lorenz system input current linput- Here lext — —0.15, Amp ~ 0.002, 
and K = 2 ■ 10^. Note the different scales for linput and Vm{t). 
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Figure 13: Pjsi{A) for R15 model neuron output when a scaled x{t) signal 
from the Lorenz system is presented with I^xt — —0.15. The entropy of this 
distribution iJ(A) = 5.3. 
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Figure 14: ISI Reconstruction of the input Lorenz signal to an R15 neuron. The 
solid line is the actual input to the neuron. The dots joined by dashed lines are 
the ISI reconstructions. The embedding dimension of the reconstruction is 
7, the time delay T is 1, lext — —0.15, k is 2 • 10^, and a linear map was used. 
The RMS error of the estimates over 10,000 estimations is cr = 0.0094 and the 
maximum error is about 0.03. 
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